Synchronizing image captures in multiple sensor devices

ABSTRACT

A method of synchronizing images from multiple image sensors having different properties is described. The method comprises synchronizing a capture of a first image using a first image sensor and a capture of a second image using a second image sensor; and storing at least one of (a) the captured first image on an on-sensor memory of the first image sensor, or (b) the captured second image on an on-sensor memory of the second image sensor.

FIELD OF THE INVENTION

An embodiment of the invention relates generally to devices that capturean image, and in particular, to a method of synchronizing images frommultiple image sensors having different properties.

BACKGROUND OF THE INVENTION

For multi-camera systems using CMOS camera sensor technology in typicalconsumer electronics form factors, such as the smartphone for example,mechanical shutters could be used for synchronization. Mechanicalshutters allow the sensors to be exposed globally at the same time.However, mechanical shutters are expensive, bulky, failure prone, andalso not useful for capturing video.

Some CMOS sensors offer a global shutter mode, where additionalcircuitry is able to hold charge temporarily. However, such CMOS sensorstypically have higher noise and poorer image quality. In a multi-camerasystem, it is also possible to slow down all the sensors to match to theslowest camera. Such a modification may make the system too slowoverall, may also make dynamic adjustment of camera parametersdifficult, and may cause both system slow-downs and frame drops. Someexisting solutions may also address both parallax and object motion byusing a processing unit receiving the image data. By trying to solvethis ill-posed problem, image quality may be sacrificed.

Even if a frame start is synchronized during the capture of multipleimages, the rest of the image may not be captured synchronously inconventional devices, where they may rely on best effortsynchronization.

Accordingly, there is a need for improved synchronization of image datain multi-camera systems.

SUMMARY OF THE INVENTION

A method of synchronizing images from multiple image sensors havingdifferent properties is described. The method comprises implementing afirst image sensor configured to capture a first image; implementing asecond image sensor configured to capture a second image; synchronizingthe capture of the first image and the second image; and storingcaptured image data for at least one of the first image sensor and thesecond image sensor on a memory of the at least one of the first imagesensor and the second image sensor.

An electronic device is also described. The electronic device comprisesa first image sensor configured to capture a first image; a second imagesensor configured to capture a second image; and a processor, whereinthe processor: synchronizes the capture of the first image and thesecond image; and stores captured image data for at least one of thefirst image sensor and the second image sensor on a memory of the atleast one of the first image sensor and the second image sensor

A non-transitory computer-readable storage medium having data storedtherein is further described, where the data represents instructionsexecutable by a processor to perform a method comprising synchronizing acapture of a first image using a first image sensor and a capture of asecond image using a second image sensor; and storing at least one of(a) the captured first image on an on-sensor memory of the first imagesensor, or (b) the captured second image on an on-sensor memory of thesecond image sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an exemplary block diagram of an electronic device havingmultiple cameras;

FIG. 2 is an exemplary diagram showing the capture of image data usingdual cameras that are matched;

FIG. 3 is an exemplary diagram showing timing differences in capturingimage data using wide angle and tele cameras;

FIG. 4 is an exemplary diagram showing the effects of the use ofmultiple cameras having different properties on an image;

FIG. 5 is an exemplary block diagram of a circuit for synthesizingimages from multiple cameras having different properties;

FIG. 6 is an exemplary block diagram showing different interfacesassociated with a pixel array and a memory of a camera;

FIG. 7 is an exemplary diagram of showing the capture and read out ofimage data associated with multiple camera having different properties;

FIG. 8 is an exemplary diagram showing the capture of image data atcorresponding times using multiple cameras having different properties;and

FIG. 9 is an exemplary flow diagram showing a method of synchronizingimages from multiple sensors having different properties.

DETAILED DESCRIPTION OF THE DRAWINGS

Multi-camera systems may often be used in high quality consumer digitalphotography, where dual cameras in mobile phones are commonly used.Synchronizing the image capture of 2 or more heterogeneous cameras iscritical for many applications, such as applications requiring differentresolutions or focal lengths, or where rolling shutters in CMOSapplications complicate the synchronization processing.

The circuits and methods set forth below describe CMOS image sensorshaving memory that allow the sensors to perform an analog-to-digital(A/D) conversion at much higher rates than supported by interface speedsassociated with other portions of the circuit enabling image capture,such as an interface to a processing unit. Additionally, DRAM can beintegrated on the sensors with high bandwidth local interfaces. UsingDRAM memory and fast A/D conversion in the local interface, multiplecamera systems can be synchronized more effectively. By performing A/Dconversion at a rate that matches the fastest sensor in the system (i.e.based upon field-of-view captured per unit time), saving the results inon-sensor DRAM, and transmitting the saved data on-demand over slowerinterfaces (which may be shared by multiple cameras), the circuits andmethods circumvent the limitation of existing interfaces and allows foreffective time synchronization of image captures. By synchronizing imagecaptures in multi-camera devices, the circuits and methods set forthbelow allow processing units, also generally known as processors, toonly correct for parallax, and therefore not have to correct for bothparallax and motion in an ill-posed situation.

While the specification includes claims defining the features of one ormore implementations of the invention that are regarded as novel, it isbelieved that the circuits and methods will be better understood from aconsideration of the description in conjunction with the drawings. Whilevarious circuits and methods are disclosed, it is to be understood thatthe circuits and methods are merely exemplary of the inventivearrangements, which can be embodied in various forms. Therefore,specific structural and functional details disclosed within thisspecification are not to be interpreted as limiting, but merely as abasis for the claims and as a representative basis for teaching oneskilled in the art to variously employ the inventive arrangements invirtually any appropriately detailed structure. Further, the terms andphrases used herein are not intended to be limiting, but rather toprovide an understandable description of the circuits and methods.

Turning first to FIG. 1, a block diagram of an electronic device havingmultiple cameras is shown. The exemplary electronic device 100 that maybe any type of device having multiple cameras. The mobile device 100 maycomprise a processor 102 coupled to a plurality of cameras 104 and 105.While cameras 104 and 105 are shown, it should be understood that thecameras comprise image sensors, and that the cameras may be independentof each other or may share circuitry. The mobile device 100 could be anytype of device adapted to transmit and receive information, such as asmart phone, tablet or other electronic device receiving or providinginformation, such as a wearable device. The processor circuit 102 couldbe an ARM processor, an X86 processor, a MIPS processor, a graphicsprocessing unit (GPU), a general purpose GPU, or any other processorconfigured to execute instructions stored in a memory. The processorcircuit 102 could be implemented in one or more processing devices,where the processors may be different. For example, the electronicdevice could include a central processing unit (CPU) as well as a GPUfor example.

The processor 102 may be coupled to a display 106 for displayinginformation to a user. The processor 102 may also be coupled to a memory108 that allows storing information related to data or informationassociated with achieving a goal. The memory 108 could be implemented asa part of the processor 102, or could be implemented in addition to anycache memory of the processor, as is well known. The memory 108 couldinclude any type of memory, such as a solid state drive (SSD), Flashmemory, Read Only Memory (ROM) or any other memory element that provideslong term memory, where the memory could be any type of internal memoryof the electronic drive or external memory accessible by the electronicdevice.

A user interface 110 is also provided to enable a user to both inputdata and receive data. Some activity tracking may require user's manualinput. The user interface could include a touch screen user interfacecommonly used on a portable communication device, such as a smart phone,smart watch or tablet computer, and other input/output (I/O) elements,such as a speaker and a microphone. The user interface could alsocomprise devices for inputting or outputting data that could be attachedto the mobile device by way of an electrical connector, or by way of awireless connection, such as a Bluetooth or a Near Field Communication(NFC) connection.

The processor 102 may also be coupled to other elements that receiveinput data or provide data, including various sensors 111, an inertialmeasurement unit (IMU) 112 and a Global Positioning System (GPS) device113 for activity tracking. For example, an inertial measurement unit(IMU) 112 can provide various information related to the motion ororientation of the device, while GPS 113 provides location informationassociated with the device. The sensors, which may be a part of orcoupled to a mobile device, may include by way of example a lightintensity (e.g. ambient light or UV light) sensor, a proximity sensor,an environmental temperature sensor, a humidity sensor, a heart ratedetection sensor, a galvanic skin response sensor, a skin temperaturesensor, a barometer, a speedometer, an altimeter, a magnetometer, a hallsensor, a gyroscope, WiFi transceiver, or any other sensor that mayprovide information related to achieving a goal. The processor circuit102 may receive input data by way of an input/output (I/O) port 114 or atransceiver 116 coupled to an antenna 118.

Turning now to FIG. 2, a diagram shows the capture of image data usingdual cameras that are matched. While two cameras are shown by way ofexample in the figures below, it should be understood that additionalcameras could be implemented according to the various circuits andmethods described. Images from multiple cameras which are matched (i.e.have similar properties) are easily combined. However, the use ofmultiple cameras having different properties enables improving thequality of images, such as by providing improved low light capability,optical zoom or other features. Images from multiple cameras arecombined using different algorithms to improve the quality over an imageprovided by a single-camera system.

Conventional products using CMOS sensors with Electronic RollingShutters (ERS) are limited in the exposure and read-out of image datafrom camera sensor to processing unit. CMOS sensors with ERS must exposesensor pixels line-by-line and control exposure digitally. As a result,there is never one “instant” of photo capture, but rather the top partof the image is exposed first, and the bottom part of image is exposedseveral milliseconds later (for example approximately 30 ms later). Thetime difference between the top line and the bottom line of the imagemay be constrained by different factors, including analog-to-digitalconversion throughput and interface speed between a camera sensor and aprocessing unit for example.

In addition to ERS, multi-camera systems also have to account forparallax, which is a result of the cameras not having aligned opticalaxes. That is, the different cameras have a baseline between them, anddifferent objects in the scene at different depths are observed by thecameras with a slightly different point of view, causing a parallaxeffect in the output images. A downstream processing unit (PU) must beable to handle parallax when it combines images from multiple cameras toproduce one unified image.

In multi-camera scenarios where the sensor characteristics are different(e.g., different megapixels, different frame rates, different focallengths, etc.), synchronized image capture can mitigate many imagequality (IQ) problems in downstream processing. However, due to ERS, itis not sufficient to synchronize the start of exposure and read-out.Even if the top left pixel in the image sensor array startssynchronized, later pixels may be read out of sync due to differences insensor characteristics. As shown for example in FIG. 3, it takes alonger time t2 to read out a portion of the tele image (the entire line)than the time t1 to read out the corresponding portion of the wideimage. A consequence of the timing mismatch is shown in FIG. 4, wherethe relative motion of a car due to lack of synchronization may changean estimation by the processing unit of the disparity or depth. Thattiming mismatch may cause downstream problems in a variety ofapplications (e.g. W/T fusion, portrait mode, etc.).

Turning now to FIG. 5, a block diagram of a circuit for synthesizingimages from multiple cameras having different properties is shown. Thecircuit of FIG. 5 comprises a first image sensor 502 and a second imagesensor 504. Each of the image sensors 502 and 504 comprises a pixelarray for storing image data (i.e. pixel values associated with animage), and a memory 508 coupled receive the image data from the pixelarray 506 and control signals from a control circuit 510. The memory iscoupled to a pixel interface 512 that provides image data to aprocessing unit 514.

Even if a CMOS sensor of a pixel array were equipped with differentcircuitry to control the ERS rate and an attempt is made to match theworst-case sensor, the sensor must transmit pixel data to the processingunit faster than the interface to the processing unit may be able toreceive the pixel data. For example, in the case of a dual camera havinga wide angle (often referred to as wide or W) sensor and telephoto orzoom (referred to as tele or T) sensor with a 2× focal lengthdifference, the central quarter of a line of the wide image must be readout at the same speed as a full line of the tele image for perfectsynchronization. As a result, it is necessary that the tele camerainterface must be able to support 4x the speed of the wide angle sensor,which may not be possible or practical.

If the tele image sensor interface cannot support the 4× speed of thewide image sensor camera interface, the processing unit must be equippedto process image data from multiple sensors that is out of sync, and beable to address such synchronization problems. However, this requiresthe processing unit to address parallax problem as well assynchronization problems. That is, the processing unit must determine ifdifferences between the images from a multi-camera system is due to lackof synchronization (e.g., moving object) or due to parallax. As such,this is an ill-posed problem that may be difficult for the processingunit to solve. For perfect discrimination of parallax or object motion,either parallax should be zero (i.e. the cameras are optically aligned,which is not feasible in the form factor of consumer electronic devices,or lab scenes at fixed depth only, which is not practical), or camerasneed to be perfectly synchronized. If the cameras can somehow besynchronized for the entirety of the image, the processing unit only hasto handle parallax problem in multi-camera systems, which is a moreconstrained problem to solve than the combination of the synchronizationand parallax problems. The circuit of FIG. 5 having memory on the imagesensors 502 and 504 enables synchronization of the two images detectedby different image sensors, as will be described in more detail below.

As shown in FIG. 6, a block diagram shows different interfacesassociated with a pixel array and a memory of a camera. Moreparticularly, a lens 602 provides data to pixel array 506 under thecontrol of the control circuit 510. An A/D converter 604 enables theconversion of analog values associated with pixels of an image intodigital values that can then be stored into the on-sensor memory 508,which may be a dynamic random access memory (DRAM) for example. Thepixel interface 606 enables the transfer of image data to the processingunit 514.

The implementation of a sensor as shown in FIG. 6 performs camerasynchronization in multi-camera systems by using fast A/D converters andDRAM on camera sensors and eliminates or significantly reduces motionbased differences in outputs of cameras in a multi-camera system. Thiscamera synchronization allows downstream processing to focus onaddressing parallax and therefore not having to solve the ill-posedproblem of handling parallax as well as well as moving object analysis.

Turning now to FIG. 7, a diagram that shows the capture and read out ofimage data associated with multiple cameras having different propertiesand describes the operation of synchronizing the images using thecircuits of FIGS. 5 and 6. There are many challenges that are a resultof using different image sensors, such as a wide angle image sensor or atele image sensor. The FOVs for image sensors having differentproperties often may not line up in time, such as when differences inexposure times may exist due to different sensor characteristics. A rowreset may be synchronized using sync/trigger circuitry of the processingunit and software program delays. While it is possible that the read outof wide angle FOV can be slowed down to match tele FOV readout, thiswould require dynamic FOV adjustment on the wide image, such as duringzoom transitions, which may be problematic due to delay required inadjusting sensor side crops, and can also negatively impact video framerate. The interface from camera to processing unit may be dedicated orshared between multiple cameras using a multiplexer. The processingunit, having time-synchronized image captures, only needs to calculateparallax.

In one embodiment of the invention, each camera is able to configureinternal FOV, and read-out speed (i.e. pixel array to memory, which maybe 1/120 s per frame for tele camera having one half the FOV of the widecamera running at 1/30 s per frame) that is much faster than interfacespeed to transfer data from the memory to the processor (which may be1/30 s per frame). The processing unit configures each image sensor bytransmitting control signals and triggers to the image sensors 502 and504 as set forth above. Each image sensor uses the configurationinformation to readout pixel arrays into memory, allowing image captureto be synchronized. Subsequently, pixel information is transmitted tothe processing unit at a slower speed as shown in FIG. 7. That is, thetriggering of the tele sensor is set to time t3 to ensure the alignmentof rows of the pixels of the tele image to correspond to rows of thepixels in the center of the wide image as shown.

As shown in FIG. 7, each camera uses the configuration information toreadout pixel arrays into memory, allowing image capture to besynchronized. Subsequently, pixel information is transmitted to theprocessing unit at slower speed. When the processing unit now hastime-synchronized image captures, the processing unit only needs tocalculate parallax. While the wide image sensor stores image dataincluding the left and right pixels (not stored by the tele imagesensor) as shown in FIG. 8, the image data associated with the centerhalf of the wide image corresponds to the full line of the tele image,where t5 and t6 of FIG. 8 corresponds to the readout time of a singlerow. When corresponding data of both the wide and tele images arestored, they can be read out at a rate that can be accommodated by theprocessing unit (e.g. the pixel interface 606), which may be slower thanthe rate of storing image data in the memory of the sensor (e.g. the A/Dinterface 604).

In the example of FIGS. 7 and 8 having a 2× optical zoom with awide-tele configuration, a tele camera readout is speeded up to matchthe matching FOV of a wide angle image. For this example, this requiresa 4× increase in read out speed for of the tele image compared with thewide image, which typically may not be handled by the interface speed tothe processor. Therefore, a 2× vertical speed-up on as shown in FIG. 7,and a 2× horizontal speedup with blanking inserted to line up pixelswithin a line, as shown in FIG. 8. DRAM contents are then read out ofthe processor at a slower rate as shown in FIG. 7. Any additional zoomfactor that reduces FOV on both the wide angle and tele image can behandled in the system as is normally done in conventional devices.

Therefore, the invention performs camera synchronization in multi-camerasystems by using fast A/D converters and DRAM on the camera sensors andeliminates or significantly reduces motion based differences in outputsof cameras in a multi-camera system. Each camera is able to configureinternal FOV, read-out speed (to memory), blanking, and is also able toaccept external trigger to synchronize captures, where A/D conversionand readout speed (Pixel Array to Memory) is much faster than theinterface speed (typically 1/120 s (or higher) vs 1/30 s per frame) ofthe processing unit. For example, the processing unit configures eachcamera/sensor with synchronization related information and sends capturetrigger to each camera to synchronize start of capture. FOV, delay andblanking info is calculated to minimize or eliminate time differencebetween captures from multiple image sensors.

Turning now to FIG. 9, a flow diagram shows a method of synchronizingimages from multiple sensors having different properties. A first imagesensor, such as image sensor 502, configured to capture a first image isimplemented at a block 902. A second image sensor, such as image sensor504, configured to capture a second image is implemented at a block 904.A synchronization configuration for the first image sensor and thesecond sensor is calculated at a block 906. The capture of the firstimage and the second image is synchronized at a block 906. The captureof the first image and the second image could be synchronized using theprocessing unit 514 as described above. Captured image data for at leastone of the first image sensor and the second image sensor is stored on amemory, such as on-sensor memory 508, of the at least one of the firstimage sensor and the second image sensor at a block 910. The capturedimages are transferred to a processor at a block 912.

The various elements of the methods of FIG. 9 may be implemented usingthe circuits of FIGS. 1-8 as described, or using some other suitablecircuits. While specific elements of the method are described, it shouldbe understood that additional elements of the method, or additionaldetails related to the elements, could be implemented according to thedisclosure of FIGS. 1-8.

It should be noted that other variations could be implemented indifferent implementations. For example, there may be more than 2 camerasin the system. One or more cameras in the system may not have memory andbe the baseline camera to which other cameras are synchronized.Depending on camera configurations, some cameras may use high speedinterface to read out directly to the processor if bandwidth allows. Theprocessing unit may have dedicated point-to-point pixel interfaces withthe camera sensors or through some shared resource, such as amultiplexer switch. An embodiment of the invention provides a method tosynchronize image capture in a CMOS sensor based multi-camera systemsthat must rely on ERS for image capture. In the examples below, a dualcamera case is described with Wide-Tele configuration (with 2× opticalzoom), but the embodiment of the invention is easily extendable to morethan two cameras of different characteristics such as differentresolutions or FOVs.

It can therefore be appreciated that new circuits for and methods ofsynchronizing images from multiple sensors having different propertieshave been described. It will be appreciated by those skilled in the artthat numerous alternatives and equivalents will be seen to exist thatincorporate the disclosed invention. As a result, the invention is notto be limited by the foregoing implementations, but only by thefollowing claims.

I claim:
 1. A method of synchronizing images from multiple image sensorshaving different properties, the method comprising: synchronizing acapture of a first image using a first image sensor and a capture of asecond image using a second image sensor; and storing at least one of(a) the captured first image on an on-sensor memory of the first imagesensor, or (b) the captured second image on an on-sensor memory of thesecond image sensor.
 2. The method of claim 1 wherein storing at leastone of (a) the captured first image on an on-sensor memory of the firstimage sensor, or (b) the captured second image on an on-sensor memory ofthe second image sensor comprises storing captured image data on theon-sensor memory of the first image sensor and the on-sensor memory ofthe second image sensor.
 3. The method of claim 1 wherein a read-outspeed of pixel values from a pixel array to the on-sensor memory of thefirst image sensor or the second image sensor is adjusted to match aread-out speed of the matching field of view of a pixel array on secondimage sensor.
 4. The method of claim 1 wherein synchronizing the captureof the first image and the second image comprises triggering the firstimage sensor to capture the first image at a first time and triggeringthe second image sensor to capture the second image at a second time. 5.The method of claim 1 wherein storing at least one of (a) the capturedfirst image on an on-sensor memory of the first image sensor, or (b) thecaptured second image on an on-sensor memory of the second image sensorcomprises storing the at least one the first captured image or thesecond captured image using a first interface at a first rate, themethod further comprising transferring the stored first captured imageor the second captured image from the on-sensor memory at a second ratethat is slower than the first rate.
 6. The method of claim 1 whereinimage data of a full line of a frame is read out of the first imagesensor in the same time as image data for a portion of the frame thatcorresponds with the field of view of the full line of the first imagesensor is read out of the second image sensor.
 7. An electronic device,comprising: a first image sensor configured to capture a first image; asecond image sensor configured to capture a second image; and aprocessor coupled to the first image sensor and the second image sensor,wherein the processor: synchronizes the capture of the first image andthe second image; and stores captured image data for at least one of thefirst image sensor and the second image sensor on a memory of the atleast one of the first image sensor and the second image sensor.
 8. Theelectronic device of claim 7 wherein storing at least one of (a) thecaptured first image on an on-sensor memory of the first image sensor,or (b) the captured second image on an on-sensor memory of the secondimage sensor comprises storing captured image data on the on-sensormemory of the first image sensor and the on-sensor memory of the secondimage sensor.
 9. The electronic device of claim 7 wherein a read-outspeed of pixel values from a pixel array to the on-sensor memory of thefirst image sensor or the second image sensor is adjusted to match aread-out speed of the matching field of view of a pixel array on secondimage sensor.
 10. The electronic device of claim 7 wherein synchronizingthe capture of the first image and the second image comprises triggeringthe first image sensor to capture the first image at a first time andtriggering the second image sensor to capture the second image at asecond time.
 11. The electronic device of claim 7 wherein storing atleast one of (a) the captured first image on an on-sensor memory of thefirst image sensor, or (b) the captured second image on an on-sensormemory of the second image sensor comprises storing the at least one thefirst captured image or the second captured image using a firstinterface at a first rate, the method further comprising transferringthe stored first captured image or the second captured image from theon-sensor memory at a second rate that is slower than the first rate.12. The electronic device of claim 7 wherein image data of a full lineof a frame is read out of the first image sensor in the same time asimage data for a portion of the frame that corresponds with the field ofview of the full line of the first image sensor is read out of thesecond image sensor.
 13. The electronic device of claim 12 wherein theprocessor further calculates parallax based upon image data from thefirst image sensor and the second image sensor.
 14. A non-transitorycomputer-readable storage medium having data stored therein representinginstructions executable by a processor to perform a method comprising:synchronizing a capture of a first image using a first image sensor anda capture of a second image using a second image sensor; and storing atleast one of (a) the captured first image on an on-sensor memory of thefirst image sensor, or (b) the captured second image on an on-sensormemory of the second image sensor.
 15. The non-transitorycomputer-readable storage medium of claim 14 wherein storing at leastone of (a) the captured first image on an on-sensor memory of the firstimage sensor, or (b) the captured second image on an on-sensor memory ofthe second image sensor comprises storing captured image data on theon-sensor memory of the first image sensor and on the on-sensor memoryof the second image sensor.
 16. The non-transitory computer-readablestorage medium of claim 14 wherein a read-out speed of pixel values froma pixel array to the on-sensor memory of the first image sensor or thesecond image sensor is adjusted to match a read-out speed of thematching field of view of a pixel array on second image sensor.
 17. Thenon-transitory computer-readable storage medium of claim 14 whereinsynchronizing the capture of the first image and the second imagecomprises triggering the first image sensor to capture the first imageat a first time and triggering the second image sensor to capture thesecond image at a second time.
 18. The non-transitory computer-readablestorage medium of claim 14 wherein storing at least one of (a) thecaptured first image on an on-sensor memory of the first image sensor,or (b) the captured second image on an on-sensor memory of the secondimage sensor comprises storing the at least one the first captured imageor the second captured image using a first interface at a first rate,the method further comprising transferring the stored first capturedimage or the second captured image from the on-sensor memory at a secondrate that is slower than the first rate.
 19. The non-transitorycomputer-readable storage medium of claim 14 wherein image data of afull line of a frame is read out of the first image sensor in the sametime as image data for a portion of the frame that corresponds with thefield of view of the full line of the first image sensor is read out ofthe second image sensor.
 20. The non-transitory computer-readablestorage medium of claim 19 further comprising calculating parallax basedupon image data from the first image sensor and the second image sensor.